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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 20 
January 2006 has been entered. 

Response to Arguments 

Applicant's arguments, see Remarks and amendments to the claims, filed 20 
January 2006, with respect to the rejection(s) of claim(s) 1-28 under 35 USC 103(a) 
have been fully considered and are persuasive. 

Therefore, the rejection of claims 1-28 has been withdrawn in view of applicant's 
amendments to the claims. 

Claims 2, 4, 6, 13-16, 18, 21-23, and 25 have been canceled. 

However, upon further consideration, a new ground(s) of rejection is made in 
view of various references as below. 

Examiner takes issue with applicant's blanket assertion that support for all 
elements within the amended claims find their support in the specification. A bit more 
information is required to make that assertion. Applicant should at least provide the 
approximate location in the originally filed specification where the amended elements 
have support. A blanket statement such as the above is only a broad, condusory 
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statement and cannot be regarded as having either factual or probative value (see In re 
Geisler) since arguments and statements of counsel are not given the above weight. 

Claim Refections - 35 USC § 101 
35 U.S.C. 101 reads as follows: 

Whoarer invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may abtam a patent therefor, subject to the 
conditions and requirements of this title. 

Claim 26 stands rejected under 35 U.S.C. 101 because it is directed to a 
computer program per se, which is the quintessential abstract idea. See the Interim 
Guidelines for Patent Subject Matter Eligibility. It further does not meet the standards 
set forth by In re Lowry or In re Beauregard for properly claiming a computer program 
product. 

Claim 27 is rejected under 35 USC 101 as unpatentable because it is directed to 
a computer program per se. A computer program must be must both functionally and 
structurally interrelated to the underlying medium upon which it is placed to be 
statutory (see In re Warmerdam). The claimed 'storage medium' is not functionally 
inten-elated, since the computer does not execute the method of claim 21 . The claim 
should be worded such that a computer Is required to execute the method. Next, the 
storage medium must be computer-readable in order for it to be executed in the first 
place. Since the preamble of the method to daim 21 does not specify that the method 
is computer-implemented the claim can also be regarded as inoperable because 
there is no mechanism by which mechanism exists to read the computer and then 
execute it 
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In order for the above claim to be statutory, it needs to recite that the "storage 
medium" is computer-readable and that the program stored thereon is for making a 
computer execute the program. Otherwise, such an emk>odiment would amount to 
functional descriptive material on a non-functional medium (e.g. computer code written 
on a piece of pap)er) that could not be transferred to a machine to cause it operate in 
some desired manner (see In re Alappat and In re Warmerdam). They further fiail to 
generate concrete, tangible, and practical applications, since a computer program per 
se is certainly not tengible and concrete, and a 'storage medium' that cannot be used by 
a computer to execute the method (or that merely contains data of an any sort) lacks 
both a 'specific, substantial, and credible' utility, and certainly does rK)t provide any 
practical benefit, and furthemiore would not be concrete in the sense that the CAFC 
detennined the tenn in State Sf.) 

Therefore, both of the above claims are prima fade nonstatutory. 

The claims rejected above under 35 USC 101 as nonstatutory are further 
rejected as set forth below in order to expedite prosecution in the expectation that 
applicant will amend the claims to place them within one of the four statutory categories 
of invention. 

Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it. in such full, dear, concise, and exact terms as to enable any person skilled in the 
art to w4iich it pertains, or with wAiich it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated tiy the inventor of carrying out his invention. 
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Claims 24 and 26-28 are rejected under 35 U.S.C. 112. first paragraph, as based 
on a disclosure that is not enabling. A computer, which is critical or essential to the 
practice of the invention, but not included in the claim(s) is not enabled by the 
disclosure. See In re Mayhew, 527 F.2d 1229. 188 USPQ 356 (CCPA 1976). That is, 
the method requires a computer in order to practice it (see for example element 210 in 
Figure 2. element 310 in Rgure 3. and the like, within the instant specification, which 
dearly illustrate that the image pickup device (camera) is clearly connected to a 
computer or computer-based unit, which therefore clearly requires such for the method 
as well. The preamble of these claims (independent) must be amended to read 'A 
computer-implemented method for 

Claims 1-28 are rejected under 35 USC 112, first paragraph, as based on a 
disclosure that is not enabling. Simply put, the claims require 'an image pick-up device 
having a fixed positional relationship with respect to a measurement object'. This 
recitation has no support in the specification -eee for example claims Figures 2 and 3, 
where the invention is embodied in a system with a HMD (head-mounted display) on a 
user who is moving around. The display would contain both an attitude sensor and a 
camera. This camera would in fact inherently not have a 'fixed positional relationship' 
with respect to a measurement object A moving user simply does not remain in a fixed 
position. The recitation only has meaning in an instantaneous context, that is, when the 
attitude and position sensors take a snapshot at a particular instant in time, ttie camera 
will have a fixed relationship to the object it is viewing only for that instant of sampled. 
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discrete time. However, the recitation in the claim does not recite that. It only states 
that the fixed positional relationship exists. Dictionary.com defines 'fixed' as: 

1 (a) To place securely; make stable of firm: fixed the tent poles in the ground 
2(a) To put into a stable or unalterable fonn: tried to fix the conversation in her 
memory 

3 To direct steadily: fixed her eyes on the road ahead 
5(a) To set or place definitely; establish: fixed her residence 
5(b) To determine with accuracy; ascertain: fixed the date of the ancient 
artifacts 

As noted above, definition 5(b) appears to be the dosed to the intended 
meaning. The instant spedfication and/or intrinsic record does not support the meaning 
of the term Tixed' in this context, thusly examiner turns to a dictionary to detennine a 
proper meaning. This meaning is not inconsistent with the disdosure of the instant 
spedfication, because the invention is enabled for a system with a moving user as 
discussed above. Therefore, the above rejection is entirely consistent with the multi- 
part analysis and reliance on both intrinsic and external records as per AWH v. Phillips 
(CAFC 2005). 

However, examiner does note that the instant specification does set forth that for 
certain situations, particulariy outdoors, the distance from the observer to the target 
objed is suffidently large that a certain distance is chosen as the fixed distance' and 
that number is used, since the calculations can tolerate a certain amount of error. 
Therefore, in order to corred the above problem, applicant needs to amend the first 
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clause of the independent claims to add Vith limited precision' to make clear that the 
fixed position relationship is in fact being approximated based on less than the best 
available infomnation to simplify computation and the like. As originally written, the first 
clause of claim 1 is not dear on this point. 

As currently written, the independent claims recite generic limitations (fixed 
positional relationship') that is only enabled for one particular embodiment, ttiat of the 
approximated position with a certain amount of error, as described in the instant 
specification. The CAFC decided LizardTech in 2005, which held that a daim reciting a 
method tiiat had scope over and covered all method of generating seamless DWT 
images was not enabling and was indefinite because the specification only described 
one embodiment that did not enable the generic daim for generating all seamless DWT 
images. Therefore, applicant must amend tiie claims in order for them not to be struck 
down by tiie Board or the CAFC. 

The following is a quotation of tiie second paragraph of 35 U.S.C. 112: 

The specification shaO conclude with one or more daims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 1 and 24 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to parfa'cularty point out and distinctiy daim tiie subject matter which 
applicant regards as tiie invention. 

Specifically, applicant uses tiie temiinology 'adapted to' in tiie independent 
daims, which has been held by the CAFC to be boUi vague and indefinite. This 
tenninology means that tiie system is not required to perform Uiat fijndionality, but 
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rather that it only may have that form, function, or required functionality. As such, any 
limitations that use the "adapted to' language are prima facie indefinite and therefore 
can be ignored in construing the claims. 

The terms 'adapted to' are exemplary language that requires three things: 1 ) that 
there be sufficient structure to provide for the language [in this case, the instant claims 
^il because they do not provide suffident structure to provide for the language's 
presence], 2) that the scope of the invention be dear [in this case, it is not, because it is 
unclear whether or the daimed feature is critical; thusly, any element without criticality is 
optional under that above language; the specification is not enabling for those elements 
as optional], and 3) that the fundional language is supported by the specification [in the 
instant case, it is not; the specification dearly requires that the system have all of the 
above components as units or steps; their omission would NOT necessarily perform the 
same fundion as before]. 

Claims 3, 5, 7-12, 17, and 19 are rejeded as not correcting the defidendes of 
their parent daim(s). 

Additionally, daims 24 and 28 are illustrative of the fad that the elements are not 
optional but are required, since there is a one4o-one mapping of elements from the 
system to the method in the instant daims. 

Claims 1 and 24 are further rejeded under 35 (JSC 112, second paragraph, 
because the words fixed positional relationship' are found to be indefinite. The 
specification does not provide a dear standard for the meaning of this terminology, and 
it is therefore indefinite because the fixed positional relationship' could be between the 
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camera and the object only during one phase of measurement, and such a relationship 
must always have a frame of reference (e.g. specific location of both object and camera, 
specific distance between two points, or the like), where this is not provided by the 
various elements in the claims. 

Further, examiner submits that the recitation of the fixed positional relationship' 
renders the daim indefinite because It constitutes a mixing of a system claim and a 
method of using that system, thusly creating undue confusion as to what precisely the 
requirements for the system daim should be. 

Definitions 

Note that the tenfn 'index* in the daims below was defined in the instant 

specification (and in the PGPub for the instant application, US 2004/0090444) in the 

following statement: 

[0051] The corrected value operation process is basically 
performed on the basis of the observation prediction position of 
the landmark on the image predicted from the sensor output and 
the observation position on the image of the landmark actually 
detected by the image processing, employing a landmark (e.g., a 
real object (or a part) that can use the features of an image 
for its projected image as an index of alignment such as the 
corner of the building or the roof of the house) . Accordingly, 
it is the maximal point of the corrected value operation process 
how to detect the observation position of landmark from the 
image correctly and stably. 



Thusly, the term 'index* in reality means 'index of alignment' which further means 
a portion or segment of the base image that is to be used for template matching (e.g. 
the comer of a house). 
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The term 'roll angle' Is never used in the specification. However, the 'roll 
direction' is referred to in the specification, where the image is rotated through a 
rotational angle to correct it It is assumed that the discussion in 4:20-5:2 of the Instant 
specification is the controlling definition. 

It is requested that applicant clarify the support in the instant specification 
for the term 'roll angle' so that examiner can determine whether or not the 
amendment in fact complies with 35 USC 112, first paragraph, with respect to the 
written description requirement A lack of complete response will result in all 
claims being rejected under 35 USC 112, first paragraph for new matter. 

/fffe/prefal/oiis 

The independent claims are not enabled and/or suffer scope of enablement • 
issues. As such, the phrase fixed positional relationship' is being interpreted as set 
forth in the specification (see PGPub [0046]). That is, the position of the other object 
can be assumed to be fixed when the distance is great enough and otherwise the 
determination of position for the measurement object can be ignored if the position of 
the measurement object is known, etc. 

Claim Reactions - 35 LflSO § 101 
35 U.S.C. 10>c]|ads as follows: 

Whoever invents or discOTe(s any new araKxsefiil process, machine, manufacture, or composition of 
matter, or any new and useftlMmprovepfi^ thereof, may obtain a patent therefor, subject to the 
conditions and requirements of f 

^ Jj"^ . 3 Claim 26 is rejected upder 35 UlS^. 1 01 because it is directed to non-statutory 

vr \ subject matter. Claim ^is directed to a compbt^r program per se. It needs to be 

combined with 27 in order to be statutory. The^t^ms rejected above under 35 
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use 101 ashqnstatptory are further rejected as set forth below in anticipation of 
applicant am^ndingil:^ daim to place it within one of the four statutory categories of 
subject^rrtatter. 

Claim Rejections • 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth In this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or descnbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sidll in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

Claims 1, 9. 17, 19-20. 24, and 26-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Sundareswaran et al (US 6,330,356) in view of Kim et al 
(5,933,530) and Chen (5,765,561). 

[Method (daim 24) and system (claim 1 ) are properiy subject to the same 
rejection when identical, and the computer program product claims are identical to the 
method claim (daim 24) and are executed by a computer, which therefore means that 
they are same (daims 26-27)] 

As to daims 1, 24, and 26-27, 
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An image processing device comprising: (Preamble is not given patentable weight, 
since it only recites a summary of the daim and/or an intended use, and the process 
steps and/or apparatus components are capable of standing on their own; see Rowe v. 
Dror, 112 F.3d 473, 42 USPQ2d 1550 (Fed. Cir. 1997), Pitney Bowes, Inc. v. Hewlett- 
Packard Co., 182 F.3d 1298, 1305, 51 USPQ2d 1161, 1165 (Fed. Cir. 1999), and the 
like.) 

-An image pick-up device having a fixed positional relationship with respect to a 
measurement object; (Sundareswaran Rgures 1 and 10, object 24 has a fixed position 
with respect to camera 20. at least in the context of the instant specification and its 
meaning - see 5:5-35 for head-mounted camera on user, where the display is head- 
mounted as well (5:49-54). Note next 8:55-65, which clearly states: "The actual camera 
position and orientation is not directly 'known' by (accessible to) the module, except as 
a current best estimate..." This is clearly comparable to the technique used in the 
instant specification, as described and discussed in the above rejection of daim 1 under 
35 use 1 12, first paragraph, as lacking enablement.) 

-An orientation sensor adapted to measure the orientation at an image pick-up visual 
point of said image pick-up device; (Sundareswaran Figure 1, 5:20-30, the system uses 
the camera to track positions of various portions of the object (e.g. natural features of 
the object - 5:10-12). Tliese are used to measure the orientation of the device, 
spedfically (abstract, 4:5-35))(eh^p/l^cy;K6/C0luriHT^lJ^^ 
P>^rji<£l(ar1^^^ndj0^^ 
an]^tpmiral^stfDs$u^^.6D^«Qd.a'ti^e^^ 
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-A storage unit adapted to store calculation Information to calculate the orientation of 
said measurement object or the position of said measurement object on the basis of an 
output from said orientation sensor; (Sundareswaran inherently requires this, since the 
system maintains registration between a virtual model and the actual position of the 
camera on the user with respect to the object - see for example 4:1 0-21 , where such a 
storage device is part of the image processing host 30 as shown in Figures 1 and 10. 
Next, note 6:35-65, where a potential application is described in detail, particulariy 
illustrating that once the position is known, it is continually tracked to maintain 
registration, which therefore a priori requires the storage of the previous position and 
orientation (Abstract, 4:10-21 , 8:55-65, etc). The output from the orientation sensor is 
used to establish the position, as it tracks the position of landmarks or fiducial marks 
(which as noted above can be natural and inherently part of the object)) 
-A prediction position calculation unit adapted to calculate a prediction position of an 
index in an image picked-up by said image pick-up device on the basis of the measured 
orientation; (Sundareswaran deariy teaches in Figure 9, 6:35-60, 8:57-65, 10:57-65, 
that the coordinate values of the predicted feature positions are used. Note step 250 in 
Figure 9, where the position is predicted 'Compute Difference Vector between Predicted 
and Measured Marker Positions'. This ttierefore requires tiiat ttie position be predicted 
based on the measured position. Note 8:57-65, where only Uie current best estimate of 
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camera position is provided to the module to perform registration and generation of the 
3D module from the virtijal camera, tiiusly the predicted position of the virtual camera is 
based on the basis of the previously measured orientation as aboye){(i^(i^jp^^Ci^J^ 
lin^s^;l^^^4^_d}S€lQs^^mafrixJtaf^^ 
toi(ie^.dj6N^ase/€QiJb©<p^ 

St(^l^dlQj}rfn^tQ2S.^9^S^^ 

l^^^^^^Qfi^/QlitByt'f^8^nJb^^^£^^ 

-A target image-setting unit adapted to extinct an image area of tiie index from tiie 
picked-up image on tiie basis of ttie prediction position of ttie index, rotate tiie extracted 
image area using a roll angle, according to the measured orientation, of said image 
pick-up device, and output tiie rotated image area as a target image; (Sundareswaran 
5:20-23, 8:57-67, ietc-tiie mariner or feature locations are extracted and their positions 
are used to determine tiie orientation of tiie object - where tiiese extracted markers are 
cleariy tiie 'index* referred to as in tiie 'definitions' section, and tiiis reference is 
specifically enabled for using natural features of an object. Cleariy tiie positions of such 
features are detemnined, the system would output the overall location of tiiat mariner 
image ... see for example 10:57-67, where tiie feature positions are the mariner 
locations suitably extiacted by tiie algorithm. Several things are calculated based on 
tills - botii T and Q (rotation mati-ix) - 10:35-40, which cleariy shows tiiat the mariner 
positions are extiacted and rotated by Q, and then output to determine T, the position 
matrix) 
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-A detecting unit adapted to detect the position of said index in said target image by 
performing a template matching process between a template image of said index and 
said target image; (Sundareswaran clearly teaches that pattern matching can be used 
for this kind of applications, such as in 2:39-55, and in Figure 4 provides an example of 
a template for a marker. In 8:25-30, Sundareswaran mentions that template matching is 
an option but is currently not used because of computational cost) 
-An updating unit adapted to update said calculation information stored in said storage 
unit on the basis of a detected position of said index detected by said detecting unit; 
(Sundareswaran clearly teaches in 10:55-67 that the position and orientation 
infomnation are updated -6:65-7:5. where the storage unit is inherent as described 
above)(Cl^en/5b(um^^ 

■H^TforpFi^feM;^^ — 



patient 




Xkn^fT^yalue^lijTd^^ 
-A calculation unit adapted to calculate the orientation and/or position of said 
measurement object on the basis of a measured value from said orientation sensor and 
said calculation information updated by said updating unit (Sundareswaran clearly 
calculates the changed position, see Rgure 7, where the position is calculated based on 
the updated information, where the process loops endlessly to update the tracking 
position and create the new best estimate as discussed above, where Figure 9 also 
shows this in step 256 ('Update 3D camera parameters (position, orientation)))(|| 
TBt>pnc^tiT^irtuanrn^e gerfel'qte^.tyaQiagegener^r SOtTa^ 
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in registration with the nsd^imade generated by camera AS^^e virtifal camera posHrbn 
will be known relatK^ to the victual anatomical strucUire, and hence the real camera 
position will be known relative to the real anatomical structure. Thus, real matnx 
M.sub.PQ^rili also be knowifi. In addition, sjrice M.sub.CT and lyi.sub.PC are then both 

/ 

knowpC it is possible to solvjb for M.sub.Rt. Accordingly, the pojbition of anatomical 



7 

tructu 



stracture 60 will then also oe knowp/within the relative coordinate sy^m defined by the 
racker system.') 



Sundareswaran teaches all the limitations of the instant claim except actually 
using template matching to perform the pattern matching, but rather uses an edge 
detection system that approximates template matching at a lower computational cost. 
The background of Sundareswaran implicitly teaches that template matching is the 
preferable method but for the increased computation cost involved. The system of Kim 
et al provides a more efficient method of template matching that only extracts a small 
area and performs the regional comparison in real time so that the cost overhead 
present in the use of a template matching technique (the computational cost as above) 
is therefore not an issue (9:1-40), which is an improvement over previously used 
template matching region (Abstract). Therefore, since this provides a simplified 
template matching algorithm which can operate in real time, which algorithms are 
known to be more efficient (see the Background of Sundareswaran), it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Sundareswaran to utilize template matching instead of edge detection, particulariy when 
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using natural features of a recognition object instead of the drcular fidudal marks of the 
preferred embodiment, since an arbitrarily-shaped mark would need template matching 
in order to be located. The system of Kim operates in a real-time manner, which 
therefore does change the principle of operation of the system of Sundareswaran. 

Sundareswaran does not expressly teach having an orientation sensor per se on 
the image pick-up device. The system of Chen provides thait having such a sensor is 
very important (10:30-1 1:5) for maintaining registration accuracy during a surgical 
procedure (which would also be critical during repairs on complex machinery and the 
like, as discussed in Sundareswaran). As such, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Sundareswaran to 
have such an orientation sensor, if only as a back up to provide additional information 
under certain circumstances, since it is critical to always have such infomiation as per 
Chen. Chen also utilizes fiducial mariners to maintain orientation and registration, and 
as such is analogous art with and to Sundareswaran. Finally, Sundareswaran is being 
combined with some of the teachings of Chen (the orientation sensor and certain details 
of calculating) to hold registration accuracy and reduce registration error. It should be 
noted that the system of Sundareswaran recalculates position after updating it, which 
therefore takes care of the last clause of the daim. 

As to daim 9, 

Chen et al. disdoses the device of daim 9. Column 1 1 , lines 12-29, disdoses 
calculation information in the form of matrix transfonnations for calculating ttie position 
and attitude of tiie patient on the basis of tiie measured value and position information 
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of the pick-up visual point of the camera in regard to the tracker system. "In this setting, 
M.sub.PC can be considered to represent the matrix transformation from the patient's 
anatomical structure 60 to camera 45; M.sub.CT can be considered to represent the 
matrix transfomiation from camera 45 to tracker base 75; and M.sub.PT can be 
considered to represent the matrix transformation from anatomical structure 60 to 
tracker base 75. 

M.sub.CT is known from the tracker system. Furthemriore, once the virtual image 
generated by image generator 30 has been placed in registration with the real image 
generated by camera 45, the virtual camera position will be known relative to the virtual 
anatomical structure, and hence the real camera position will be known relative to the 
real anatomical structure. Thus, real matrix M.sub.PC will also be known. In addition, 
since M.sub.CT and M.sub.PC are then both known, it is possible to solve for M.sub.PT. 
Accordingly, the position of anatomical structure 60 will then also be known within the 
relative coordinate system defined by the tracker system." Therefore, the calculation 
information is updated once the virtual image is placed in registration with the real 
image generated by the camera and the matrix transformations are fonmed. 

As to claim 17, Sundareswaran teaches that the measurement object is an image 
pickup visual point (see rejection to daim 19 below and 5:5-55), where the display unit 
is a HUD and overlays the virtual image on the real world in accordance with the 
determined position and orientation so as to maintain registration between the real 
world and the superimposed images in the virtual space using the calculation unit as 
described in the rejection to daim 1 above. 
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As to claim 19, Sundareswaran clearly teaches a HUD (heads-up display) that a 
user can wear with the camera t>eing placed on the user's head (5:5-55). This therefore 
provides that the measurement object Is visual point of the observer, and the virtual 
image is always overlaid with the real world image as required by the calculation image, 
since registration is maintained between them, and the image is clearly optically 
transmitted in the real space through the display screen as described above. 

As to claims 20 and 28, these are duplicates of claims 1 and 24 in some ways. It 
omits various elements of claims 1 and 24. The CCPA ruled In re Karison that the 
omission of an element and its function in combination is an obvious expedient if the 
remaining elements perform the same functions as before. In re Karison (CCPA) 136 
USPQ 184 (1963). The omitted elements are; a storage unit, an updating unit, and a 
calculation unit. Method and system and subject to the same rejection as explained 
above. 

Claims 3, 5. 7-8, and 10-12 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Sundareswaran, Kim, and Chen as applied to daim 1 and further in 
view of Raliison et al. 

As to claim 3, 

Sundareswaran, Kim, and Chen disclose the device of daim 3 except wherein 
said calculation information is the correction information to conrect for an error in the 
measured value of attitude measured by said attitude sensor of the image pick-up visual 
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point of said image pick-up device, and said calculation unit calculates attitude of said 
measurement object on the basis of the measured value and correction information. 
The invention of Railison et al. teaches of tracking the position and attitude of an image 
generator mounted on the head of a person. Column 1 9, lines 1 1 - 26, teaches of the 
various types of tracker systems including magnetic and inertial sensor systems. 
Columns 21 , 22, and 23 explain obtaining position and attitude orientation with the two 
systems. Column 24, lines 37 - 67, and column 25, lines 23 - 62, describe correcting 
for an error in the measured values of attitude and position taken from the sensor 
systems. It would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify the invention of Sundareswaran and Chen to include 
using an inertial sensor system as the tracker system for the video camera. One would 
have been motivated to make such a modification to Sundareswaran since inertial 
sensors such as rate gyros will have the advantage of being substantially immune to 
magnetic perturi3ations that may be present in the area around machinery. It would 
have been further obvious to one having ordinary skill in the art at the time the invention 
was made to correct for an error in the measured value of attitude information of the 
Inertial sensors for use in the matrix transformation calculations of Sundareswaran. 
One would have been motivated to make such a modification to the invention of 
Sundareswaran so that the calculated position and attitude information of the camera 
and patient remain accurate while perfbrming a repair on complex machinery. 
As to claims 10, 11, and 12, 
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Chen et al. discloses the device of claims 10, 1 1 , and 12. Column 1 1 , lines 12 - 
29, discloses calculation infonnation in the fomi of matrix transformations for calculating 
the position and attitude of the patient on the basis of the measured value and position 
information of the pick-up visual point of the camera in regard to the tracker system. "In 
this setting, M.sub.PC can be considered to represent the matrix transfonnation from 
the patient's anatomical structure 60 to camera 45; M.sub.CT can be considered to 
represent the matrix transfonnation from camera 45 to tracker base 75; and M.sub.PT 
can be considered to represent the matrix transformation from anatomical structure 60 
to tracker base 75. 

M.sub.CT is known from the tracker system. Furthennore, once the virtual image 
generated by image generator 30 has been placed in registration with the real image 
generated by camera 45, the virtual camera position will be known relative to the virtual 
anatomical structure, and hence the real camera position will be known relative to the 
real anatomical structure. Thus, real matrix M.sub.PC will also be known. In addition, 
since M.sub.CT and M.sub.PC are then both known, it is possible to solve for M.sub.PT. 
Accordingly, the position of anatomical structure 60 will then also be known within the 
relative coordinate system defined by the tracker system." Therefore, the calculation 
infonnation is updated once the virtual image is placed in registration with the real 
image generated by the camera and the matrix transformations are formed. 
Additionally, column 12, lines 35 - 37, describes using a least squares fit to con-elate 
the sampled points with the patient-spedfic images. Thus, a typical value is used that 
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incorporates a dislocation value between the predicted position and the detected 
position of a plurality of markers from a real and virtual image. 
As to claim 5, 

Sundareswaran, Kim, and Chen disclose the invention of claim 5 except wherein 
said calculation infbnnation is the correction irrformation to con-ect for an error in the 
measured value of attitude measured by said attitude sensor and the position 
information of the image pick-up visual point of said image pick-up device, and said 
calculation unit calculates the position and attitude of said measurement object on the 
basis of the measured value, correction information, and the position information. The 
invention of Rallison et al. teaches of tracking the position and attitude of an image 
generator mounted on the head of a person. Column 1 9, lines 1 1 - 26, teaches of the 
various types of tracker systems including magnetic and inertial sensor systems. 
Columns 21. 22, and 23 explain obtaining position orientation with the two systems. 
Column 24, lines 37 - 67, and column 25, lines 23 - 62, describe correcting for an enx)r 
in the measured values of attitude and position taken from the sensor systems. It would 
have been obvious to one having ordinary skill in the art at the time the invention was 
made to modify the invention of Chen et al. to include using an inertial sensor system as 
the tracker system for the video camera. One would have been motivated to make such 
a modification to Chen since inertial sensors such as rate gyros will have the advantage 
of being substantially immune to magnetic perturbations that may be present in a 
medical operating atmosphere. It would have been further obvious to one having 
ordinary skill in the art at the time the invention was made to correct fbr an error in tiie 
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measured value of attitude and position Information of the Inertia! sensors for use in the 
matrix transformation calculations of Chen. One would have been motivated to make 
such a modification to the invention of Chen so that the calculated position and attitude 
information of the camera and patient remain accurate while perfomnlng a medical 
procedure. 

As to daim 7, 

Sundareswaran, Kim. Chen, and Ralllson teach of the device of claim 7. 
Sundareswaran, Kim, Chen in view of Rallison teach of updating the position 
Information with regard to a series of matrix transfonnatlons in column 11, lines 7 - 30. 
Additionally, column 9, lines 38-49, describe a search algorithm for automatically 
bringing a virtual image into registration with a real image. Column 10, lines 38 - 52, 
describes reestablishing the registration between a real image and a virtual image if the 
video camera position is moved. Thus, Sundareswaran, Kim, Chen, and Rallison teach 
of updating position Information. It would have been obvious to one having ordinary skill 
in the art at the time the invention was made to modify the invention of Sundareswaran, 
Kim, Chen, and Rallison so that the position infonnation Is only updated in two 
directions and not in a depth direction if the action is so desired by a user while 
reestablishing a registration between a real Image and a virtual image. One would have 
been motivated to make such a modification to the invention of Sundareswaran, Kim, 
Chen, and Rallison so that If a user does not require an updated value in a depth 
direction, the computing system does not waste utilization time and resources 
calculating an unnecessary value. 
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As to claim 8, 

Sundareswaran. Kim, Chen, and Rallison explicitly disclose the device of claim 8 
except wherein the correction information is the Information to correct for an error In the 
azimuth direction among the measured values of the attitude measured by the attitude 
sensor. In column 23, lines 60 -67, and column 24, lines 1-23, Rallison additionally 
teaches of sensor drift, thus causing an error in the measured values of roll, pitch, and 
yaw. 

The definition of the term VaW is "to turn about the vertical axis. Used of an 
aircraft, spacecraft, projectile, etc." The definition of the tenn "azimuth" is "the horizontal 
angular distance firom a reference direction, to the point where a vertical circle through a 
celestial body intersects the horizon, usually measured clockwise. Sometimes the 
southern point Is used as the reference direction, and the measurement is made 
clockwise through 360 degrees." 

Therefore, the amount of yaw is the amount of turn around a vertical axis, while 
the azimuth is the amount of turn around a reference direction. The two terms are 
synonymous in this context, e.g. the amount of yaw deviation, drift, or error, is 
equivalent to amount of azimuth deviation, drift, or error. 

Thus, it is inherent that the use of a gyroscopic tracking system in the invention 
of Sundareswaran, Kim, and Chen in view of Rallison will accumulate an error in the 
measured directions, including the azimuth measurements. Therefore, the use of the 
yaw correction information in the matrix transfomiations in the invention of 
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Sundareswaran, Kim. and Chen in view of Rallison Is equivalent to the use of the 
azimuth correction Information. 

Conclusion 

The prior art made of record and not relied upon Is considered pertinent to 
applicant's disclosure. Note US 5.878,156 to Okumura, which generates template 
images In real time so that the computational issues Involved in normal template 
matching do not occur and that it makes template matching faster than the edge 
detection. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eric Woods whose telephone number is 671-272-7775. 
The examiner can normally be reached on M-F 7:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor. Ulka Chauhan can be reached on 571-272-7782. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Infonmation Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status infomiation for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated infonnation 
system, call 800-786-91 99 (IN USA OR CANADA) or 571-272-1 000. 

Eric Woods May 23, 2006 
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